System and method for automated grain inspection during harvest

ABSTRACT

A system and method for automated grain inspection and analysis of results during harvest, using an inspection system mounted on a combine harvester with geolocation tracking, allowing for real time analysis during harvest and tracking of grain quality by location of harvest.

CROSS-REFERENCE TO RELATED APPLICATIONS

Priority is claimed in the application data sheet to the followingpatents or patent applications, the entire written description of eachof which is expressly incorporated herein by reference in its entirety:

-   -   Ser. No. 17/522,161    -   Ser. No. 16/436,592    -   62/605,957    -   62/606,332    -   Ser. No. 16/122,853    -   Ser. No. 16/434,497

BACKGROUND Field of the Art

The disclosure relates to the field of image analysis, and moreparticularly to the field of using image analysis to automaticallyinspect and analyze grains (seeds and pulses) during harvest.

Discussion of the State of the Art

Grains inspections and related applications for quality control, processcontrol, food safety and grading for commercial value are based onsubjective measures, use human interpretation of the inspected objectswith pictures provided and descriptive specifications provided by thestandards bodies.

Inspection of various grains (for example, various plant grains such aswheat or rice, mineral or metallic grains, or granulated or powderedsubstances) for various purposes such as safety or marketability isgenerally limited by factors such as subjectivity and speed, due toreliance on manual inspection methods. These methods also do not scalewell and thus inspection is restricted to a sample group that is assumedto be an accurate representation of the entire lot, and study has shownvisual inspection to have an error rate of 20-30%. Further, the grainsmust usually be sent to a laboratory for inspection, resulting in delaysin inspection, and not allowing for tracking of the location at whichthe grains were harvested.

What is needed is a system and method for automated grain inspection andanalysis of results during harvest, using an inspection system mountedon a combine harvester with geolocation tracking, allowing for real timeanalysis during harvest and tracking of grain quality by location ofharvest.

SUMMARY

Accordingly, the inventor has conceived and reduced to practice, asystem and method for automated grain inspection and analysis of resultsduring harvest, using an inspection system mounted on a combineharvester with geolocation tracking, allowing for real time analysisduring harvest and tracking of grain quality by location of harvest.

According to a preferred embodiment, a system for automated food safetyanalysis, quality analysis, and grading of grains, comprising: an imageprocessor, comprising at least a plurality of programming instructionsstored in the memory of, and operating on at least one processor of, acomputing device, wherein the plurality of programming instructions,when operating on the processor, cause the computing device to: receivea digital image of grains; identify and count within the image the areasassociated with individual grains; extract dimension information foreach individual grain identified; create, for each individual grainidentified, a pixel map of the color data for each pixel within the areaof the image associated with that individual grain; and transmit orstore the data comprising individual grain count, dimension information,and pixel map for each individual grain for analysis; and a food safetyanalyzer, comprising at least a plurality of programming instructionsstored in the memory of, and operating on at least one processor of, acomputing device, wherein the plurality of programming instructions,when operating on the processor, cause the computing device to: receiveor obtain data from the image processor for a sufficient number ofimages from a single lot of grain to constitute a statisticallyrepresentative sample for the lot of grain; compare each pixel map inthe data against pixel maps from reference images of infected grains ofthe type being inspected; perform a food safety analysis, based on thepixel map comparisons, comprising at least the type and extent ofinfection for each individual grain and the percentage of infectedgrains in the data for the statistically representative sample; andcompare the results of the food safety analysis against at least onepre-defined standard for assessing food safety; and provide acertificate of analysis for the lot of grain detailing the extent towhich the lot of grain meets the at least one pre-defined standard forassessing food safety; and a quality analyzer, comprising at least aplurality of programming instructions stored in the memory of, andoperating on at least one processor of, a computing device, wherein theplurality of programming instructions, when operating on the processor,cause the computing device to: device to: receive or obtain data fromthe image processor for a sufficient number of images from a single lotof grain to constitute a statistically representative sample for the lotof grain; compare each pixel map in the data against pixel maps fromreference images of damaged grains of the type being inspected; performa quality analysis, based on the pixel map comparisons, comprising atleast the type and extent of damage for each individual grain and thepercentage of damaged grains in the data for the statisticallyrepresentative sample; and compare the results of the quality analysisagainst at least one industry standard for assessing grain quality; andprovide a certificate of analysis for the lot of grain detailing theextent to which the lot of grain meets the at least one industrystandard for assessing grain quality, is disclosed.

According to another preferred embodiment, a method for automated foodsafety analysis, quality analysis, and grading of grains, comprising thesteps of: receiving, at an image processor, comprising at least aplurality of programming instructions stored in the memory of, andoperating on at least one processor of, a computing device, a digitalimage of grains; identifying and counting within the image the areasassociated with individual grains; extracting dimension information foreach individual grain identified; creating, for each individual grainidentified, a pixel map of the color data for each pixel within the areaof the image associated with that individual grain; transmitting orstore the data comprising individual grain count, dimension information,and pixel map for each individual grain for analysis; receiving orobtaining, at a food safety analyzer, comprising at least a plurality ofprogramming instructions stored in the memory of, and operating on atleast one processor of, a computing device, data from the imageprocessor for a sufficient number of images from a single lot of grainto constitute a statistically representative sample for the lot ofgrain; comparing each pixel map in the data against pixel maps fromreference images of infected grains of the type being inspected;performing a food safety analysis, based on the pixel map comparisons,comprising at least the type and extent of infection for each individualgrain and the percentage of infected grains in the data for thestatistically representative sample; comparing the results of the foodsafety analysis against at least one pre-defined standard for assessingfood safety; providing a certificate of analysis for the lot of graindetailing the extent to which the lot of grain meets the at least onepre-defined standard for assessing food safety; receiving or obtaining,at a quality analyzer, comprising at least a plurality of programminginstructions stored in the memory of, and operating on at least oneprocessor of, a computing device, data from the image processor for asufficient number of images from a single lot of grain to constitute astatistically representative sample for the lot of grain; comparing eachpixel map in the data against pixel maps from reference images ofdamaged grains of the type being inspected; performing a qualityanalysis, based on the pixel map comparisons, comprising at least thetype and extent of damage for each individual grain and the percentageof damaged grains in the data for the statistically representativesample; comparing the results of the quality analysis against at leastone industry standard for assessing grain quality; and providing acertificate of analysis for the lot of grain detailing the extent towhich the lot of grain meets the at least one industry standard forassessing grain quality, is disclosed.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

The accompanying drawings illustrate several aspects and, together withthe description, serve to explain the principles of the inventionaccording to the aspects. It will be appreciated by one skilled in theart that the particular arrangements illustrated in the drawings aremerely exemplary, and are not to be considered as limiting of the scopeof the invention or the claims herein in any way.

FIG. 1 is a diagram illustrating an exemplary system for automated graininspection and analysis, according to a preferred embodiment of theinvention.

FIG. 2 is a diagram illustrating an exemplary alternative systemarchitecture for automated grain inspection and analysis, according toanother embodiment of the invention.

FIG. 3 is a flow diagram illustrating an exemplary logical architecturefor automated grain analysis, according to another embodiment of theinvention.

FIG. 4 is a diagram illustrating an exemplary color wheel for use incolor calibration and analysis, according to an embodiment of theinvention.

FIG. 5 is a diagram illustrating an exemplary color analysis result,according to an embodiment of the invention.

FIG. 6 is a diagram illustrating an exemplary reporting interfacewindow, presenting report results of automated grain inspection andanalysis, according to an embodiment of the invention.

FIG. 7 shows an exemplary system for grain analysis, according to anembodiment.

FIG. 8 shows details of an exemplary operation of the system and methoddisclosed herein

FIG. 9 shows the analysis of the area and color of a grain, according toan aspect of the invention.

FIG. 10 is a block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 11 is a block diagram illustrating an exemplary logicalarchitecture for a client device.

FIG. 12 is a block diagram showing an exemplary architecturalarrangement of clients, servers, and external services.

FIG. 13 is another block diagram illustrating an exemplary hardwarearchitecture of a computing device.

FIG. 14 shows an exemplary representation of the bin structures of ABchromaticity space of grain samples.

FIG. 15 shows the process by which grains are analyzed.

FIG. 16 shows a combine harvester, as currently used for harvestingcrops.

FIG. 17 shows an example of a system with camera, light and lightcontrol, computer and cloud application.

FIG. 18 shows the workflow of the system of FIG. 17.

FIG. 19 shows a process for milling yield management.

FIG. 20 shows a software interface for milling yield management.

FIG. 21 shows an exemplary histogram of the Lab values of tabulateddata.

FIG. 22 shows data for a grain.

FIG. 23 shows an exemplary certificate of analysis.

FIG. 24 shows the workflow of the software wizard of FIG. 18.

FIG. 25 shows the workflow of the user program of FIG. 18.

FIG. 26 shows the chalky distribution graph and values.

DETAILED DESCRIPTION

The inventor has conceived, and reduced to practice, a system and methodfor automated grain inspection and analysis of results during harvest,using an inspection system mounted on a combine harvester withgeolocation tracking, allowing for real time analysis during harvest andtracking of grain quality by location of harvest.

What is needed is a system and method that measures, counts, calculates,classifies, and reports size, shape, color, color distribution, damages,unsafe properties, quality score and other important properties thatreplaces the manual and subjective interpretation methods based ondescription and pictures as reference, with accurate, measurable,repeatable, and empirical values. The ability to quantify good grains,damages, health risks, commercial grading and quality score using anabsolute and objective process and the results of such a process arecritical to the entire agriculture ecosystem and humanity in general.The results include safer food, higher quality and desired taste andtexture for the consumer. In addition, accurate data measured by arobust system can be used for further analysis, process control, yieldimprovement, event prediction, alerts, and other uses thus enablingcollaboration among the industry ecosystem to help address issues in afaster and more reliable way.

The system includes cameras to capture the image of each grain in asample, and illumination units to combine light sources with visible andinvisible wavelengths (for example, infrared or ultraviolet). Both thecameras and light controls are connected to a computer that runsoperating software and application programs. In addition, the results ofan inspection may be uploaded automatically to cloud applications thatstore and later use the data and images to run additional applicationswhere big data is required from one or more instruments in a facility ormultiple facilities, or in different steps in the processing, or stepsin the supply chain. For example, quality trends and comparisons, alertson food safety events, issue of certificates of analysis, visualizationof color distribution in grains, or other uses.

One or more different aspects may be described in the presentapplication. Further, for one or more of the aspects described herein,numerous alternative arrangements may be described; it should beappreciated that these are presented for illustrative purposes only andare not limiting of the aspects contained herein or the claims presentedherein in any way. One or more of the arrangements may be widelyapplicable to numerous aspects, as may be readily apparent from thedisclosure. In general, arrangements are described in sufficient detailto enable those skilled in the art to practice one or more of theaspects, and it should be appreciated that other arrangements may beutilized and that structural, logical, software, electrical and otherchanges may be made without departing from the scope of the particularaspects. Particular features of one or more of the aspects describedherein may be described with reference to one or more particular aspectsor figures that form a part of the present disclosure, and in which areshown, by way of illustration, specific arrangements of one or more ofthe aspects. It should be appreciated, however, that such features arenot limited to usage in the one or more particular aspects or figureswith reference to which they are described. The present disclosure isneither a literal description of all arrangements of one or more of theaspects nor a listing of features of one or more of the aspects thatmust be present in all arrangements.

Headings of sections provided in this patent application and the titleof this patent application are for convenience only, and are not to betaken as limiting the disclosure in any way.

Devices that are in communication with each other need not be incontinuous communication with each other, unless expressly specifiedotherwise. In addition, devices that are in communication with eachother may communicate directly or indirectly through one or morecommunication means or intermediaries, logical or physical.

A description of an aspect with several components in communication witheach other does not imply that all such components are required. To thecontrary, a variety of optional components may be described toillustrate a wide variety of possible aspects and in order to more fullyillustrate one or more aspects. Similarly, although process steps,method steps, algorithms or the like may be described in a sequentialorder, such processes, methods and algorithms may generally beconfigured to work in alternate orders, unless specifically stated tothe contrary. In other words, any sequence or order of steps that may bedescribed in this patent application does not, in and of itself,indicate a requirement that the steps be performed in that order. Thesteps of described processes may be performed in any order practical.Further, some steps may be performed simultaneously despite beingdescribed or implied as occurring non-simultaneously (e.g., because onestep is described after the other step). Moreover, the illustration of aprocess by its depiction in a drawing does not imply that theillustrated process is exclusive of other variations and modificationsthereto, does not imply that the illustrated process or any of its stepsare necessary to one or more of the aspects, and does not imply that theillustrated process is preferred. Also, steps are generally describedonce per aspect, but this does not mean they must occur once, or thatthey may only occur once each time a process, method, or algorithm iscarried out or executed. Some steps may be omitted in some aspects orsome occurrences, or some steps may be executed more than once in agiven aspect or occurrence.

When a single device or article is described herein, it will be readilyapparent that more than one device or article may be used in place of asingle device or article. Similarly, where more than one device orarticle is described herein, it will be readily apparent that a singledevice or article may be used in place of the more than one device orarticle.

The functionality or the features of a device may be alternativelyembodied by one or more other devices that are not explicitly describedas having such functionality or features. Thus, other aspects need notinclude the device itself

Techniques and mechanisms described or referenced herein will sometimesbe described in singular form for clarity. However, it should beappreciated that particular aspects may include multiple iterations of atechnique or multiple instantiations of a mechanism unless notedotherwise. Process descriptions or blocks in figures should beunderstood as representing modules, segments, or portions of code whichinclude one or more executable instructions for implementing specificlogical functions or steps in the process. Alternate implementations areincluded within the scope of various aspects in which, for example,functions may be executed out of order from that shown or discussed,including substantially concurrently or in reverse order, depending onthe functionality involved, as would be understood by those havingordinary skill in the art.

Definitions

The term “damage” as used herein means imperfections in grain caused byphysical factors such as heat, cold, flood, mechanical damage fromharvesting, transportation, and processing, and insect damage fromchewing, boring, or tunneling.

The term “disease” or “infection” as used herein means any of a numberof diseases and infections that affect grains, the most common of whichare various types of fungal infections.

The term “grain” or “grains” as used herein includes the grains, seeds,and pulses of plants.

The terms “lot” or “batch” as used herein mean a quantity of grain beingtransported, sold, processed, analyzed, certified, or otherwise handledor disposed of as a single unit.

The term “soundness” as used herein means the overall visual grainquality. The soundness of a particular grain is diminished by damage anddisease. Industry standards such as the Official United States Standardfor Grain set forth the factors for determining soundness of grain.

Conceptual Architecture

FIG. 1 is a diagram illustrating an exemplary system 100 for automatedgrain inspection and analysis, according to a preferred embodiment ofthe invention. According to the embodiment, a system 100 may comprise afeeder 130 configured to direct grains onto a receptacle 102, forexample using a rigid or flexible chute 130 that is angled to controlthe rate of flow onto a flat or curved receptacle 102 to maintain adesired grain density on the surface of receptacle 102. This allowsfeeder 130 to feed a limited amount of grain to be inspected ontoreceptacle 102 that can spread the grain out for proper inspection, forexample using mechanical means such as a vibratory motor 101 thatagitates receptacle 102 to distribute grains on the surface ofreceptacle 102. Receptacle 102 may also be manipulated either manuallyor automatically to improve spreading of the grains, for example theintensity of a vibrating motor 101 may be altered automatically if thegrain distribution is not within desired parameters (for example, as maybe detected using an image sensor 110). A plurality of light emitters120 may be used to project visible light onto receptacle 102 andilluminate any grains scattered thereupon, and a plurality ofcorresponding image sensors 110 may be used to capture image data byscanning receptacle 102 while illuminated. Optionally, light emitters120 and imaging sensors 110 may be tuned to various wavelengths that maylie within or outside the visible spectrum (for example, infrared orultraviolet), as is described below in FIG. 2. System 100 may beconnected to a computing device or a system of computing devices, suchas a network or local arrangement of computers and computing hardware,that may be configured to capture and analyze grains using thecomponents of system 100, as described below in FIG. 3.

FIG. 2 is a diagram illustrating an exemplary alternative system 200 forautomated grain inspection and analysis, according to another embodimentof the invention. According to the embodiment, a system 200 may comprisea plurality of light emitters 120, 220, 221 of different wavelengthschosen from the spectrum of light, for example ranging from infraredthrough ultraviolet positioned above and/or below receptacle 102 (inthis embodiment, a concave receptacle is shown to illustrate anadditional possible arrangement), to illuminate grains on the surface ofreceptacle 102 with various wavelengths of light from above and below. Aplurality of corresponding images sensors may be used to scan theilluminated grains, for example sensors tuned to capture visible lightfrequencies 110 as described above in FIG. 1, as well as sensorsconfigured to capture infrared (IR) 210 and ultraviolet (UV) 211. Inthis manner, multispectral analysis may be performed on grains to countand analyze them, providing detailed quantitative and qualitativeresults that may be presented in various visualizations and reports, asdescribed below in FIGS. 4-6.

In some cases, one of the color properties assessed in particular may bea degree of chalkiness of some or all of the grains dispersed on thesurface of receptacle 102, and more in particular afterglow effects ofsuch chalkiness (for example, multispectral illumination of the grainsmay reveal certain spectral behaviors associated with chalkiness thatmay otherwise be difficult to observe, revealing details otherwiseobscured in any one particular spectral band). Light emitters 120, 220,221 may use one or more, or a combination, of LEDs of different color,or by specialized uni-or multi-spectral halide or xenon or similardischarge lamps, or other light-emitting sources, and may be configuredas specialized uni- or multi-spectral lamps, and may optionally be usedwith any or a combination of filters to further alter the emissionspectra. During sampling of grain, lights may be sequenced as needed toachieve optimal image quality or to tune for specific image or grainfeatures, such as to highlight blemishes or examine for disease (eitherin general, or to examine for specific diseases or pathogens) or graindamage, or to classify grain type or variety, or to correlate withinformation regarding the location, methods, or other conditions of thegrain's growth, harvest, storage, transport, or processing.

In addition to cameras, other sensor types may include humidity sensors,temperature sensors, light sensors, scanners, scales, or other sensortypes, and the data from the sensors and cameras may be used to measureall the details of blemishes, diseases or any other damage to eachgrain, so the system can identify the grain type, its variety, and itsdiseases and damages. For each grain, a pixel count may be calculatedand then organized in a histogram for color and size. These histogramsmay be hierarchical and may be used to identify and help quicklycategorize grains, diseases, qualities, or any measurable metric.

All this information may be sent over a network to a server or a cloud,and compared to a reference database. Changes over time may be trackedby region, enabling companies, governments and NGOs to assess the safetyand sufficiency of the food supply and to recognize supply problemsstemming from new diseases quickly and early on.

System 100, 200 may be implemented in a combine harvester or otherharvesting or farming equipment, for example diverting a sample from aharvest stream according to a configured time, location, or otherschedule or pattern. This may be used to enable real-time (or nearreal-time) analysis of a harvest, for example to produce a harvestquality map that may be used to optimize field preparation (such as todirect the use of fertilizers or pesticides, for example) or for storageor transmission. Grain may also be classified and tracked based onharvest time, location, methods, or other such metrics, which mayoptionally be presented alongside analysis results in reporting.

FIG. 3 is a flow diagram illustrating an exemplary logical architecture300 for automated grain analysis, according to another embodiment of theinvention. According to the embodiment, an image acquisition engine 301may collect data from a plurality of image sensors 120, 220, 221 in asystem for automated grain examination. Image sensor data may then beprovided to an image processor 302 that may perform any or a combinationof image processing tasks on the data, such as (for example, includingbut not limited to) normalization, amplification, color balancing,colorization, desaturation, edge-finding or erosion, or any other imageprocessing task that may be used to improve the suitability of imageinformation for a particular desired analysis result or operationalconfiguration (for example, processing an image to expedite analysis orto reduce system load during parallelized operation). Processed imagedata may then be provided to a parametric evaluator 303 that may comparethe processed data against configured parameters for analysis, forexample to verify that the image data is within desired parameters foroptimum results. A color analyzer 304 may then be used to analyze thecolor of grains within the image data, for example according to aconfiguration file as described below with reference to FIGS. 4-5. Coloranalyzer 304 may be used to calculate a pixel count that may then beorganized in a histogram for color and size, and pixel counts andhistograms may be arranged hierarchically and used in grainclassification or visualization as described below. A grain classifier305 may be used to classify specific grains, or a sample group as awhole, for example to apply specific labels such as size or textureidentifiers or to accept/reject grains based on analysis results. Forexample, if a sufficient quantity of grains in a sample are below aconfigured quality threshold (as determined using parameter informationfrom parametric evaluator 303 and grain color information from coloranalyzer 304), then a sample may be marked as unsatisfactory. A varietyof outputs 306 may be produced for further review, storage, ortransmission, according to various aspects and implementations.Visualizations 306 a may be used to surface analysis data for humanreview in a readily-understood fashion, for example using color wheels(as shown below in FIGS. 4-5). A summary 306 b may be provided for aconcise statement of analysis results, for example for publication orquick viewing when a large number of analyses must be checked orcompared quickly. CSV reports 306 c may be produced for storage andimport into other software applications, such as for further analysis orfor storage in a database with other information (such as to maintain arepository of historical analysis results). A grain window 306 d maypresent a view of individual grains or groups of grains for manualinspection, for example if an ambiguous result requires humanintervention to validate, or to enable an additional layer of qualitycontrol by including human verification for some or all analysisoperations. A report screen 306 e may be produced to consolidateanalysis results into a human-readable interface with various analysisfactors represented for viewing, such as the exemplary interface shownbelow in FIG. 6. Images 306 f may be produced from image data usedduring analysis, for example to store “raw” or original image dataalongside the results of analysis of the same data, or to produce imagesof grain that was analyzed for publication or storage.

FIG. 7 shows an exemplary system 700 for grain analysis, according toone aspect of the system and method disclosed herein. Inlet funnel 701conducts grain samples via pipes 703 a-n onto examination plate 705,which can rotate vertically in a full circle. Motor 704 shakes plate 705to distribute the grains so a full sample can be captured by camera 702for examination and analysis, as described in greater detail below.Lights (not shown here) may be installed in system 700 to illuminategrain on plate 705 during photography by camera 702. A software programon a computer, (not shown in this figure) controlling all elements, suchas LEDs or other lights, motor to manipulate plate 705, etc. nowanalyzes the camera images of the grains, in some cases in differentilluminations from the sources discussed throughout. In some cases, ifthe grains are too bunched up, the motor 704 that can manipulate the705, for example by having a gear box, that when run in reverse,vibrates (if square) or turns (if round) the table horizontally ratherthan flip it for dumping the grains. Then the grains can be re-examined,and this may be repeated several times until a satisfactory view isachieve The grain is then dumped from plate 705 into outlet hopper 706.

FIG. 8 shows details of an exemplary operation 800 of the system andmethod disclosed herein. Grain enters the system via pipe 801 past valve802, which valve is calibrated to admit only a predefined weight orvolume of grain into funnel 803 and thence via pipes 804 a-n onto plate805.

FIG. 9 shows the analysis of the area and color of a grain 900,according to an aspect of the invention. According to the aspect, thetotal area of grain 900 may be considered to be a value of 100% of thearea, while any damage to grain 900 may be defined as a percentagethereof Grain 900 may be of a normal color according to a configuredcolor scale, or it may be of an abnormal color either according to thecolor scale or based on the analysis of the % of the surface area thatis an abnormal color (for example, there may be a defined colorthreshold above which the entire grain is deemed to be abnormal).Specific damages 901 may be identified and considered when determiningif a grain is acceptable, for example by the nature or severity of thedamage (such as a surface imperfection) or based on the portion of totalarea that is damaged (such as for color abnormalities, or cracks).

FIG. 14 shows an exemplary representation of the bin structures of ABchromaticity space 1400 of grain samples. For a sample, each grain ispresented in average, mode, and median. For the grains, each grain isrepresented by roughly 3000-6000 pixels. For a lab sample, the L range1401 is typically set to be 0-100; the AB (XY) axes 1402 a and 1402 bmay run from −100 to +100 and may be divided into 16 bins 1403 a-n, forexample. This number is somewhat arbitrary and may be changed forvarious reasons. In some cases, these numbers may be dynamically changedto maximize the number of bins (within the quadrants) 1403 a-n. To run asample, the system typically creates ten groups of L1-10 (L forlightness based on captured pixels). Each L group, typically, with 16quadrants 1403 a-n in a range of +/−100 to +/−100 AB axis 1402 a, 1402 bof chromaticity. For each L (Lxaby) group, a user places the pixel inthe relevant *ab bin 1403 a-n. The system then summarizes the number ofpixels per bin, calculates each bin as a percent of area of the samplegrain or object, and calculates the average of L-a-b for each bin. Thesystem is now able to create a plot showing LAB-bin-average values,percent of area, and actual color. For more detail, see FIG. 15 below.

The chromaticity space is used in conjunction with color visualreferences provided by various groups, industries, and governmentalagencies for identifying disease and damage in crops. For example, theUSDA visual reference library contains color photographs of a variety ofgrain defects, disease, damage, contamination, spoilage, infection, andother factors, and USDA grading tables provide quality categories basedon the color of rice kernel, milling degree, and maximum damages allowedper grade. Under current methodology, a human inspector needs tovisually compare the actual grain with the provided pictures ordescriptions as reference, and make qualitative judgment calls regardingdisease, damage, and soundness. When using such reference, there is noqualitative number or specification of the color or the minimum area ofthe “heat damaged” spot. USDA visual references and other sourcesprovide information on a variety of grain defects, disease, damage,contamination, spoilage, infection, and other factors. These visualreferences can be used to input color information into the system torecognize quantitatively, for example, mold contamination, which can bea health concern. These quantitative characterizations usingchromaticity space are far more accurate than human visual inspection,and the results are more repeatable and reliable. A wide variety ofgrain-related health issues and food safety issues can be identified inthis manner.

FIG. 15 shows the process 1420 by which grains are analyzed. In step1421 the raw data file is loaded, as shown in chart 1422. In step 1423,the kernel number is fetched and the kernel ID is then assigned. Data ofthe identified kernel is extracted from a file 1424. Then a new file iscreated 1425, containing only data for that selected kernel or objectand then the data is loaded into tables L1-L10 1426 a-n. In the nextstep, additional tables 1427 a-n are made. These tables contain theLxaby matrices. Finally, table 1427 a-n is expanded into a full table1428 a-n with mainly two values (Lab, Avg) for each pixel that then areoutput into a regular file. This file is then processed, for example inFIG. 22.

Detailed Description of Exemplary Aspects

FIG. 4 is a diagram illustrating an exemplary color wheel 400 for use incolor calibration and analysis, illustrating the use of configured zones410, 420, 430, according to an embodiment of the invention. According tovarious aspects of the invention, color analyzer 304 may receive orproduce a calibration file comprising configuration information thatdefines a plurality of color zones 410, 420, 430 on a color wheel 400,that may be used in conjunction with slices of color wheel 400 torepresent color analysis points. A calibration file may be produce byscanning and analyzing a known sample of grains and fitting thecalibration to the results, defining calibration values in terms of theresults obtained from the use of a controlled sample that is known toproduce specific values. Color zones are defined as an area covering thefull 360° of the color wheel 400 out to a specified distance from theorigin. For example, zone A 410 may be the area out to ⅓r (where r isthe radius of the circle described by color wheel 400), zone B 420 maycomprise the area from ⅓r to ⅔r, and zone C 430 may be the remainingouter area from ⅔r to r, or the outer boundary of color wheel 400. Thespecific values for each zone may be described in a calibration file forease of storage and use. It should be appreciated that the specificvisual arrangement shown in FIG. 4 may vary, for example orienting thecolor wheel with 0° at the bottom, side, or at an angle (rather than atthe top as shown), or increasing the degree scale in a counterclockwisedirection, rather than the clockwise direction shown, or othervariations (which may be defined in a calibration file).

FIG. 5 is a diagram illustrating an exemplary color wheel 500 for use incolor calibration and analysis, illustrating the use of configured colorslices 510, 520, 530, 540, according to an embodiment of the invention.According to various aspects, color slices 510, 520, 530, 540 may bedefined in a configuration file (optionally in addition to , with eachslice comprising a circular sector within color wheel 500. In thismanner, color analysis results may be represented as points within colorwheel 500, with each point being placed within the slice and/or zonecorresponding to the analysis results, producing a completevisualization for easy interpretation and further use of analysisresults. This may be used both to visualize actual color (for example,when using visible-light analysis of grains, as described above in FIG.1), as well as to visualize multidimensional data by assigning color toother data values. For example, grain size or degree of chalkiness maybe represented as distance from the origin (and thus, placement withincolor zones 410, 411, 412) and grain density (or other metric) may berepresented as a point's placement within color zones 510, 520, 530,540. This may be particularly suitable for some visualization types, asa normal color wheel provides for easy visual indication of color hueand saturation, which may be natural analogues for certain analysismetrics such as chalkiness, size, damage, density, purity, or any of anumber of metrics that may be represented on a bounded scale.Additionally, any particular visualization may have selectable ordynamically-adjustable granularity, for example to enable a zoom featureto precisely compare multiple points that may be grouped together,providing for a high-fidelity representation of information (forexample, by using vectors to store pixel information rather thanrounding values and thus introducing information loss).

FIG. 6 is a diagram illustrating an exemplary reporting interface window600, presenting report results of automated grain inspection andanalysis, according to an embodiment of the invention. According to theembodiment, analysis results may be collected and consolidated into areporting interface 600 for ease of viewing, so they may be presentedfor verification or review by a human user or for use in publication(such as to publish analysis results of a grain sample for publicviewing). Any number and combination of analysis metrics may berepresented for viewing in appropriate formats, for example including(but not limited to) a list of grain types 610 that were analyzed, alist of metricized analysis results 630, a graph of grain metricdistribution 620 such as color or size distribution or other metricsthat may be represented in graph form, or sortable or filterable listsof grain attributes 640 such as physical dimensions. These interfaceviews may optionally be fixed, for example as part of a loadedconfiguration or as part of a particular analysis operation (forexample, an analysis focused on specific metrics may restrict the typesof information presented for the sake of clarity), or they may beuser-configurable and interactive, for example enabling drag-and-drop orother interaction so that a user may adjust the information or thepresentation thereof

FIG. 16 shows a combine harvester 1600, as currently used for harvestingcrops. In some cases, the system and method disclosed herein may beimplemented in such a combine or other harvesting system. Implementationmay comprise occasionally diverting, for example based on schedule orlocation, a sample for analysis from a harvest stream. Such a system ofdiversion and analysis during harvesting operations could enable a userto create a near real-time harvest quality map, which may then becommunicated to any desired destinations, thus permitting farm managersto optimize field preparation for the next season. Such preparation mayfor example include, but is not limited to, the use of fertilizers orpesticides where applicable.

Kernel analyzer system 1604 is integrated into combine harvest 1600(typically near or on a driver cabin 1601) by having a sample pull 1603that allows periodic pulling of samples from a main kernel feed 1602, asample return 1606, and a link 1605 that either connect to the combine'sown network or connects via a wireless uplink directly to the Internet,or both.

Additionally, kernel analyzer system 1604 analyzes and calculates thesorter “rejected grains bin” and provides information related to returnon investment (ROI) to recover “good” grains from the rejected bin. Inaddition, it provides feedback to the sorter machine on its performance,how many good kernels are rejected on each bad kernel, and by inspectingthe grains before the sorter station as an input to the sorter foroptimization of speed vs. performance. This method can recover goodgrains that can be sold at a premium compared to damaged grains.

FIG. 17 shows an example of a system with camera 1701, light and lightcontrol 1702, computer 1703 and cloud application 1704. This system canmeasure grains of any kind from one grain to any amount that can beplaced on a working area, for example 1500-2000 grains of rice(approximately 25 grams), or several thousand grains of Quinoa(approximately 10 grams), or other such uses.

FIG. 18 shows an exemplary workflow of the system of FIG. 17. This flowincludes system operating software, graphic user interface, inputs, andoutputs of application software. Wizard 1801 is a software program thatuses a template file 1802 as input for the measurement basic setup andconfiguration of default parameters. A sample of grains (not shown) maybe placed on a working area and the user progresses through all thesteps of wizard 1801 to set all the parameters of the measurement, andat the end of the process a new calibration file is created.

The calibration file can be for rice variety (for example, basmati orjasmine), for size (long, medium or short grain), for type (rough orpaddy, brown, milled), or for special rice such as Wild or Arborio. Inadditional it may be for a specific quality standard (for example,Japan, Korea, USA) or by customer requirement either for tighterspecifications such as rice grains for sake preparation or loose qualityrequirement when grains are to be used in food processing such as forrice crackers or pet food, or because of specific weather conditionswhen further adjustments are required. Wizard 1801 steps cover all grainproperties and definitions to classify each grain as “good”, “bad”(damaged), or “questionable”, by color and dimensions (length, width,area, and calculations such as length/width ratio). All calibrations maybe stored in a folder 1803 and used for testing matching relevantsamples.

User program 1804 performs the actual measurements, classification andreporting. It receives as input a grain sample placed on a working area,and a calibration file that matches the grain type and testrequirements. The outputs are summary results, visual classification ofall the grains (for example, “broken”, “chalky”, “red”, etc.) andprovides tools to further review the results, such as a window where auser may click on a specific grain to view all the grain properties.

A report page 1805 provides additional information such as quantity ofgrain, distribution, types of grains in the sample, classification ofall damaged grains, and statistics such as (for example), number ofgrains, number of broken grains, average length, or average width. Inaddition, a test results folder 1806 may be generated and saved, withthe folder name including date and time plus optional text if entered bythe user as a sample name. Files provide data on the sample level, grainlevel, and pixel level for each grain. Images of the sample provideinformation on the visuals, classifications, and results. Images may beused to upload an image of the sample for re-testing in case of disputeor investigation, and a daily results file may be updated to maintaindaily status. Web-based or local cloud page 1807 includes a set ofsoftware applications including (for example) alerts, visualization,reporting, trends, dashboard, and other productivity tools provided toimprove quality, yield, and food safety. Wizard 1801 enables a user toset a file name and the method to define grain type, such as by lengthor length-to-width ratio, and uses USDA or other standards as defaultswith the option to modify values. The wizard 1801 provides a method toset color value and limits (for example, maximum area to classify “reddamages” in a grain), and a summary results screen with classificationof all detected damages. In addition to the system's capability tomeasure, classify, and report grain status, it also provides a set ofapplications that bring new methods to address the grain industryresulting in better quality, safer food, higher yield, and greaterprofit.

FIG. 19 shows an exemplary process for milling yield management. Yieldmanagement is based on measurement of dimensions, color and calculationof the milling process endpoint, which means a desired milling level andoptimized yield that eliminates over- or under-milling. 1901 is a colorindex presentation of all the samples 1910 a-n. What is missing in thecurrent art is ensuring that the milling process reached its desiredyield level, not over- or under-milled; therefore, the system uses thecombination of color with rice grain dimensions.

FIG. 20 shows a software interface for milling yield management. Shownis a distribution of length 2001 and width 2002, while graph 2003presents the combination of calculated area with color. In addition, thegraph 2003 shows that there are two groups of milled rice 2004 a-b; bothgroups look good visually but one 2004 b is over-milled as determined bythe system.

FIG. 21 shows an exemplary histogram 2100 of the Lab values of tabulateddata for several grains 2110 a-n.

FIG. 22 shows exemplary color data for a single grain, showing a piechart 2200 indicating portions of color variance 2201 a-n in the grainsurface.

FIG. 23 shows an exemplary certificate of analysis 2300. Most of thetransactions in the supply chain include a quality test before shipmentand incoming inspection by the receiving party. Currently, the reportsare based on collective inputs including manual inspection results. Thesystem will generate a certificate with images and digital absolutevalues measured by instruments. This will ensure accurate and objectivereporting and reduce arguments, conflicts, and shipment returns based onsubjective interpretation. The application uses the cloud databased withthe test results, and uses templates to extract the desired text andimages to include in the report.

FIG. 24 shows an exemplary workflow of the software wizard of FIG. 18.Wizard 1801 loads a template file 1802 and grain sample 2401, calibratesusing the loaded data 2402, and then stores the calibration 1804.

FIG. 25 shows an exemplary workflow of the user program of FIG. 18. Userprogram 1804 retrieves a calibration file 2501 from a calibration store1803, and uses the calibration when measuring and analyzing 2502 a grainsample 2510 or loaded historical grain sample imagery 2520. Calibratedmeasurement results 2503 may then be stored 2504 for future use, such asfor loading and using in future operations 2520.

FIG. 26 shows an exemplary chalky distribution graph and values. Thecurrent graph 2600 shows that the distribution is not linear, andaddresses the binary method provided by USDA and other standards. Chalkis an opaque area in the rice grain that occurs most commonly whengrains are exposed to high temperatures during development. Chalky ricedecreases the value of rice because of its undesirable appearance andquality Chalky rice grains are defined by USDA (for example) if morethan 50% of the area of the grain is opaque in color. This methodprovides a subjective and biased presentation of the actual grainquality. For example, a grain can be 45% chalky and yet defined as “notchalky”, or between 51-100% and defined as “chalky”. It is a binaryclassification with no additional information that provides accuratestatus of the entire sample The wizard defines a level of color rangethat is defined as “chalky” using absolute color values and definitionsof “light chalky” and “chalky” area of the total area of the grain andlater in the user program with the measurement, classification,visualization, and reporting of the two values, in addition to thepresentation of the entire sample distribution chart.

An alert system (not shown) will detect and notify selected users andwill perform actions when triggered. User screen sets alert thresholds,for example if the amount of broken grains is higher than 5% or if thelevel of chalkiness is higher than 20%. Alerts may be via email, voicenotification, SMS, or any other electronic communication method withrelevant people or systems. A list of alerts is configured by the userand the cloud application monitor it periodically and compares with thetest results of the sample. The alert system may notify in differentlocations of activity, incoming inspection to alert on bad grains in ashipment, processing for quality results, after storage, or beforepacking. This tool is valuable to improving overall food safety,quality, and operational efficiency.

Hardware Architecture

Generally, the techniques disclosed herein may be implemented onhardware or a combination of software and hardware. For example, theymay be implemented in an operating system kernel, in a separate userprocess, in a library package bound into network applications, on aspecially constructed machine, on an application-specific integratedcircuit (ASIC), or on a network interface card.

Software/hardware hybrid implementations of at least some of the aspectsdisclosed herein may be implemented on a programmable network-residentmachine (which should be understood to include intermittently connectednetwork-aware machines) selectively activated or reconfigured by acomputer program stored in memory. Such network devices may havemultiple network interfaces that may be configured or designed toutilize different types of network communication protocols. A generalarchitecture for some of these machines may be described herein in orderto illustrate one or more exemplary means by which a given unit offunctionality may be implemented. According to specific aspects, atleast some of the features or functionalities of the various aspectsdisclosed herein may be implemented on one or more general-purposecomputers associated with one or more networks, such as for example anend-user computer system, a client computer, a network server or otherserver system, a mobile computing device (e.g., tablet computing device,mobile phone, smartphone, laptop, or other appropriate computingdevice), a consumer electronic device, a music player, or any othersuitable electronic device, router, switch, or other suitable device, orany combination thereof In at least some aspects, at least some of thefeatures or functionalities of the various aspects disclosed herein maybe implemented in one or more virtualized computing environments (e.g.,network computing clouds, virtual machines hosted on one or morephysical computing machines, or other appropriate virtual environments).

Referring now to FIG. 10, there is shown a block diagram depicting anexemplary computing device 10 suitable for implementing at least aportion of the features or functionalities disclosed herein. Computingdevice 10 may be, for example, any one of the computing machines listedin the previous paragraph, or indeed any other electronic device capableof executing software- or hardware-based instructions according to oneor more programs stored in memory. Computing device 10 may be configuredto communicate with a plurality of other computing devices, such asclients or servers, over communications networks such as a wide areanetwork a metropolitan area network, a local area network, a wirelessnetwork, the Internet, or any other network, using known protocols forsuch communication, whether wireless or wired.

In one aspect, computing device 10 includes one or more centralprocessing units (CPU) 12, one or more interfaces 15, and one or morebusses 14 (such as a peripheral component interconnect (PCI) bus). Whenacting under the control of appropriate software or firmware, CPU 12 maybe responsible for implementing specific functions associated with thefunctions of a specifically configured computing device or machine. Forexample, in at least one aspect, a computing device 10 may be configuredor designed to function as a server system utilizing CPU 12, localmemory 11 and/or remote memory 16, and interface(s) 15. In at least oneaspect, CPU 12 may be caused to perform one or more of the differenttypes of functions and/or operations under the control of softwaremodules or components, which for example, may include an operatingsystem and any appropriate applications software, drivers, and the like.

CPU 12 may include one or more processors 13 such as, for example, aprocessor from one of the Intel, ARM, Qualcomm, and AMD families ofmicroprocessors. In some aspects, processors 13 may include speciallydesigned hardware such as application-specific integrated circuits(ASICs), electrically erasable programmable read-only memories(EEPROMs), field-programmable gate arrays (FPGAs), and so forth, forcontrolling operations of computing device 10. In a particular aspect, alocal memory 11 (such as non-volatile random access memory (RAM) and/orread-only memory (ROM), including for example one or more levels ofcached memory) may also form part of CPU 12. However, there are manydifferent ways in which memory may be coupled to system 10. Memory 11may be used for a variety of purposes such as, for example, cachingand/or storing data, programming instructions, and the like. It shouldbe further appreciated that CPU 12 may be one of a variety ofsystem-on-a-chip (SOC) type hardware that may include additionalhardware such as memory or graphics processing chips, such as a QUALCOMMSNAPDRAGON™ or SAMSUNG EXYNOS™ CPU as are becoming increasingly commonin the art, such as for use in mobile devices or integrated devices.

As used herein, the term “processor” is not limited merely to thoseintegrated circuits referred to in the art as a processor, a mobileprocessor, or a microprocessor, but broadly refers to a microcontroller,a microcomputer, a programmable logic controller, anapplication-specific integrated circuit, and any other programmablecircuit.

In one aspect, interfaces 15 are provided as network interface cards(NICs). Generally, NICs control the sending and receiving of datapackets over a computer network; other types of interfaces 15 may forexample support other peripherals used with computing device 10. Amongthe interfaces that may be provided are Ethernet interfaces, frame relayinterfaces, cable interfaces, DSL interfaces, token ring interfaces,graphics interfaces, and the like. In addition, various types ofinterfaces may be provided such as, for example, universal serial bus(USB), Serial, Ethernet, FIREWIRE™, THUNDERBOLT™, PCI, parallel, radiofrequency (RF), BLUETOOTH™, near-field communications (e.g., usingnear-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fastEthernet interfaces, Gigabit Ethernet interfaces, Serial ATA (SATA) orexternal SATA (ESATA) interfaces, high-definition multimedia interface(HDMI), digital visual interface (DVI), analog or digital audiointerfaces, asynchronous transfer mode (ATM) interfaces, high-speedserial interface (HSSI) interfaces, Point of Sale (POS) interfaces,fiber data distributed interfaces (FDDIs), and the like. Generally, suchinterfaces 15 may include physical ports appropriate for communicationwith appropriate media. In some cases, they may also include anindependent processor (such as a dedicated audio or video processor, asis common in the art for high-fidelity A/V hardware interfaces) and, insome instances, volatile and/or non-volatile memory (e.g., RAM).

Although the system shown in FIG. 10 illustrates one specificarchitecture for a computing device 10 for implementing one or more ofthe aspects described herein, it is by no means the only devicearchitecture on which at least a portion of the features and techniquesdescribed herein may be implemented. For example, architectures havingone or any number of processors 13 may be used, and such processors 13may be present in a single device or distributed among any number ofdevices. In one aspect, a single processor 13 handles communications aswell as routing computations, while in other aspects a separatededicated communications processor may be provided. In various aspects,different types of features or functionalities may be implemented in asystem according to the aspect that includes a client device (such as atablet device or smartphone running client software) and server systems(such as a server system described in more detail below).

Regardless of network device configuration, the system of an aspect mayemploy one or more memories or memory modules (such as, for example,remote memory block 16 and local memory 11) configured to store data,program instructions for the general-purpose network operations, orother information relating to the functionality of the aspects describedherein (or any combinations of the above). Program instructions maycontrol execution of or comprise an operating system and/or one or moreapplications, for example. Memory 16 or memories 11, 16 may also beconfigured to store data structures, configuration data, encryptiondata, historical system operations information, or any other specific orgeneric non-program information described herein.

Because such information and program instructions may be employed toimplement one or more systems or methods described herein, at least somenetwork device aspects may include nontransitory machine-readablestorage media, which, for example, may be configured or designed tostore program instructions, state information, and the like forperforming various operations described herein. Examples of suchnontransitory machine- readable storage media include, but are notlimited to, magnetic media such as hard disks, floppy disks, andmagnetic tape; optical media such as CD-ROM disks; magneto-optical mediasuch as optical disks, and hardware devices that are speciallyconfigured to store and perform program instructions, such as read-onlymemory devices (ROM), flash memory (as is common in mobile devices andintegrated systems), solid state drives (SSD) and “hybrid SSD” storagedrives that may combine physical components of solid state and hard diskdrives in a single hardware device (as are becoming increasingly commonin the art with regard to personal computers), memristor memory, randomaccess memory (RAM), and the like. It should be appreciated that suchstorage means may be integral and non-removable (such as RAM hardwaremodules that may be soldered onto a motherboard or otherwise integratedinto an electronic device), or they may be removable such as swappableflash memory modules (such as “thumb drives” or other removable mediadesigned for rapidly exchanging physical storage devices),“hot-swappable” hard disk drives or solid state drives, removableoptical storage discs, or other such removable media, and that suchintegral and removable storage media may be utilized interchangeably.Examples of program instructions include both object code, such as maybe produced by a compiler, machine code, such as may be produced by anassembler or a linker, byte code, such as may be generated by forexample a JAVA™ compiler and may be executed using a Java virtualmachine or equivalent, or files containing higher level code that may beexecuted by the computer using an interpreter (for example, scriptswritten in Python, Perl, Ruby, Groovy, or any other scripting language).

In some aspects, systems may be implemented on a standalone computingsystem. Referring now to FIG. 11, there is shown a block diagramdepicting a typical exemplary architecture of one or more aspects orcomponents thereof on a standalone computing system. Computing device 20includes processors 21 that may run software that carry out one or morefunctions or applications of aspects, such as for example a clientapplication 24. Processors 21 may carry out computing instructions undercontrol of an operating system 22 such as, for example, a version ofMICROSOFT WINDOWS™ operating system, APPLE macOS™ or iOS™ operatingsystems, some variety of the Linux operating system, ANDROID™ operatingsystem, or the like. In many cases, one or more shared services 23 maybe operable in system 20, and may be useful for providing commonservices to client applications 24. Services 23 may for example beWINDOWS™ services, user-space common services in a Linux environment, orany other type of common service architecture used with operating system21. Input devices 28 may be of any type suitable for receiving userinput, including for example a keyboard, touchscreen, microphone (forexample, for voice input), mouse, touchpad, trackball, or anycombination thereof Output devices 27 may be of any type suitable forproviding output to one or more users, whether remote or local to system20, and may include for example one or more screens for visual output,speakers, printers, or any combination thereof Memory 25 may berandom-access memory having any structure and architecture known in theart, for use by processors 21, for example to run software. Storagedevices 26 may be any magnetic, optical, mechanical, memristor, orelectrical storage device for storage of data in digital form (such asthose described above, referring to FIG. 10). Examples of storagedevices 26 include flash memory, magnetic hard drive, CD-ROM, and/or thelike.

In some aspects, systems may be implemented on a distributed computingnetwork, such as one having any number of clients and/or servers.Referring now to FIG. 12, there is shown a block diagram depicting anexemplary architecture 30 for implementing at least a portion of asystem according to one aspect on a distributed computing network.According to the aspect, any number of clients 33 may be provided. Eachclient 33 may run software for implementing client-side portions of asystem; clients may comprise a system 20 such as that illustrated inFIG. 11. In addition, any number of servers 32 may be provided forhandling requests received from one or more clients 33. Clients 33 andservers 32 may communicate with one another via one or more electronicnetworks 31, which may be in various aspects any of the Internet, a widearea network, a mobile telephony network (such as CDMA or GSM cellularnetworks), a wireless network (such as WiFi, WiMAX, LTE, and so forth),or a local area network (or indeed any network topology known in theart; the aspect does not prefer any one network topology over anyother). Networks 31 may be implemented using any known networkprotocols, including for example wired and/or wireless protocols.

In addition, in some aspects, servers 32 may call external services 37when needed to obtain additional information, or to refer to additionaldata concerning a particular call. Communications with external services37 may take place, for example, via one or more networks 31. In variousaspects, external services 37 may comprise web-enabled services orfunctionality related to or installed on the hardware device itself. Forexample, in one aspect where client applications 24 are implemented on asmartphone or other electronic device, client applications 24 may obtaininformation stored in a server system 32 in the cloud or on an externalservice 37 deployed on one or more of a particular enterprise's oruser's premises.

In some aspects, clients 33 or servers 32 (or both) may make use of oneor more specialized services or appliances that may be deployed locallyor remotely across one or more networks 31. For example, one or moredatabases 34 may be used or referred to by one or more aspects. Itshould be understood by one having ordinary skill in the art thatdatabases 34 may be arranged in a wide variety of architectures andusing a wide variety of data access and manipulation means. For example,in various aspects one or more databases 34 may comprise a relationaldatabase system using a structured query language (SQL), while othersmay comprise an alternative data storage technology such as thosereferred to in the art as “NoSQL” (for example, HADOOP CASSANDRA™,GOOGLE BIGTABLE™, and so forth). In some aspects, variant databasearchitectures such as column-oriented databases, in-memory databases,clustered databases, distributed databases, or even flat file datarepositories may be used according to the aspect. It will be appreciatedby one having ordinary skill in the art that any combination of known orfuture database technologies may be used as appropriate, unless aspecific database technology or a specific arrangement of components isspecified for a particular aspect described herein. Moreover, it shouldbe appreciated that the term “database” as used herein may refer to aphysical database machine, a cluster of machines acting as a singledatabase system, or a logical database within an overall databasemanagement system. Unless a specific meaning is specified for a givenuse of the term “database”, it should be construed to mean any of thesesenses of the word, all of which are understood as a plain meaning ofthe term “database” by those having ordinary skill in the art.

Similarly, some aspects may make use of one or more security systems 36and configuration systems 35. Security and configuration management arecommon information technology (IT) and web functions, and some amount ofeach are generally associated with any IT or web systems. It should beunderstood by one having ordinary skill in the art that anyconfiguration or security subsystems known in the art now or in thefuture may be used in conjunction with aspects without limitation,unless a specific security 36 or configuration system 35 or approach isspecifically required by the description of any specific aspect.

FIG. 13 shows an exemplary overview of a computer system 40 as may beused in any of the various locations throughout the system. It isexemplary of any computer that may execute code to process data. Variousmodifications and changes may be made to computer system 40 withoutdeparting from the broader scope of the system and method disclosedherein. Central processor unit (CPU) 41 is connected to bus 42, to whichbus is also connected memory 43, nonvolatile memory 44, display 47,input/output (I/O) unit 48, and network interface card (NIC) 53. I/Ounit 48 may, typically, be connected to keyboard 49, pointing device 50,hard disk 52, and real-time clock 51. NIC 53 connects to network 54,which may be the Internet or a local network, which local network may ormay not have connections to the Internet. Also shown as part of system40 is power supply unit 45 connected, in this example, to a mainalternating current (AC) supply 46. Not shown are batteries that couldbe present, and many other devices and modifications that are well knownbut are not applicable to the specific novel functions of the currentsystem and method disclosed herein. It should be appreciated that someor all components illustrated may be combined, such as in variousintegrated applications, for example Qualcomm or Samsungsystem-on-a-chip (SOC) devices, or whenever it may be appropriate tocombine multiple capabilities or functions into a single hardware device(for instance, in mobile devices such as smartphones, video gameconsoles, in-vehicle computer systems such as navigation or multimediasystems in automobiles, or other integrated hardware devices).

In various aspects, functionality for implementing systems or methods ofvarious aspects may be distributed among any number of client and/orserver components. For example, various software modules may beimplemented for performing various functions in connection with thesystem of any particular aspect, and such modules may be variouslyimplemented to run on server and/or client components.

The skilled person will be aware of a range of possible modifications ofthe various aspects described above. Accordingly, the present inventionis defined by the claims and their equivalents.

What is claimed is:
 1. A system for automated grain inspection duringharvest, comprising: a sample inlet, mounted in or on a harvester, andconfigured to divert samples of grain being harvested to an imagingsystem; and the imaging system, mounted in or on the harvester, andconfigured to take digital images or video of the samples of graindiverted by the sample inlet; a humidity sensor, configured to capturemoisture data from air surrounding the samples of grain; and a sampleoutlet, mounted in or on the harvester, and configured to either returnthe samples of grain to the grain being harvested after imaging andmoisture data capture or discard the samples after imaging and moisturedata capture; and a computing device comprising a memory and aprocessor, and configured to receive the digital images or video fromthe imaging system and the moisture data from the humidity sensor; animage processor comprising a first plurality of programming instructionsstored in the memory of, and operating on the processor of, thecomputing device, wherein the first plurality of programminginstructions, when operating on the processor, causes the computingdevice to: receive the images or video of the samples of grain; identifyin each image or video a plurality of grains of the samples of grain;for each of the plurality of grains identified, determine a color valueof each pixel representing that grain; and create a histogram of thecolor values of the plurality of grains identified from the determinedcolor values; and a parametric evaluator comprising a second pluralityof programming instructions stored in the memory of, and operating onthe processor of, the computing device, wherein the second plurality ofprogramming instructions, when operating on the processor, causes thecomputing device to: receive the histogram from the image processor;determine whether the histogram falls within an expected histogramparameter; receive the moisture data from the humidity sensor; determinewhether the moisture data falls within an expected moisture parameter;and if the histogram falls within the expected histogram parameter andthe moisture data falls within the expected moisture parameter, indicateacceptability of the sample of grain; and a hierarchical histogramevaluator comprising a third plurality of programming instructionsstored in the memory of, and operating on the processor of, thecomputing device, wherein the third plurality of programminginstructions, when operating on the processor, cause the computingdevice to: if the parametric evaluator has not indicated acceptabilityof the samples of grain, receive the histogram from the parametricevaluator; and compare the histogram to a hierarchy of histograms toidentify an abnormality in the samples of grain that is a cause ofunacceptability; and a geolocation device configured to track a locationof the harvester; and a wireless communication device configured to:receive either the indication of acceptability of the samples of grainfrom the parametric evaluator or the identification of the abnormalityin the samples of grain; receive the location of the harvester from thegeolocation device; and transmit the indication of acceptability of thesamples of grain from the parametric evaluator or the identification ofthe abnormality in the samples of grain, along with the location of theharvester, wirelessly to a computer or network of computers locatedremotely from the harvester.
 2. The system of claim 1, furthercomprising the computer or network of computers located remotely fromthe harvester, configured to: receive the digital images, video, oranalyses from the wireless communication device on a plurality ofharvesters; receive location data from the wireless communication deviceon the plurality of harvesters; and track variations in grain quality bylocation of harvest.
 3. The system of claim 1, wherein the samples ofgrain diverted for inspection are held until at least one analysis isperformed by the computing device, and then either returned to the grainbeing harvested or discarded, depending on a result of the at least oneanalysis.
 4. A method for automated grain inspection during harvest,comprising the steps of: diverting samples of grain being harvested toan imaging system via a sample inlet, mounted in or on a harvester;taking digital images or video of the samples of grain diverted by thesample inlet via the imaging system, mounted in or on the harvester;capturing moisture data from air surrounding the samples of grain usinga humidity sensor; returning the samples of grain to the grain beingharvested after imaging and moisture data capture or discarding thesamples of grain after imaging and moisture data capture via a sampleoutlet, mounted in or on the harvester; receiving the digital images orvideo into an image processor operating on a computing device from theimaging system; identifying in each image or video a plurality of grainsof the samples of grain; for each of the plurality of grains identified,determining a color value of each pixel representing that grain; andcreating a histogram of the color values of the plurality of grainsidentified from the determined color values; receiving, into aparametric evaluator operating on the computing device, the histogramfrom the image processor; determining whether the histogram falls withinan expected histogram parameter; receiving the moisture data from thehumidity sensor; determining whether the moisture data falls within anexpected moisture parameter; if the histogram falls within the expectedhistogram parameter and the moisture data falls within the expectedmoisture parameter, indicating acceptability of the samples of grain; ifthe parametric evaluator has not indicated acceptability of the samplesof grain, receiving, into a hierarchical histogram evaluator operatingon the computing device, the histogram from the parametric evaluator,and comparing the histogram to a hierarchy of histograms to identify anabnormality in the samples of grain that is a cause of unacceptability;tracking a location of the harvester via a geolocation device; receivingeither the indication of acceptability of the samples of grain from theparametric evaluator or the identification of the abnormality in thesamples of grain into a wireless communication device; receiving thelocation of the harvester from the geolocation device into the wirelesscommunication device; and transmitting the indication of acceptabilityof the samples of grain from the parametric evaluator or theidentification of the abnormality in the samples of grain, along withthe location of the harvester, wirelessly from the wirelesscommunication device to a computer or network of computers locatedremotely from the harvester.
 5. The method of claim 4, furthercomprising the steps of: receiving into the computer or network ofcomputers located remotely from the harvester the digital images, video,or analyses from the wireless communication device on a plurality ofharvesters; receiving into the computer or network of computers locatedremotely from the harvester location data from the wirelesscommunication device on a plurality of harvesters; and trackingvariations in grain quality by location of harvest.
 6. The method ofclaim 4, wherein the samples of grain diverted for inspection are helduntil at least one analysis is performed by the computing device, andthen either returned to the grain being harvested or discarded,depending on a result of the at least one analysis.